Analysis of a Risk Sensitive Control

Recently considerable attention has been given to the study of risk-sensitive stochasic control problems 1]-13]. Risk-sensitive control is a generalization of the risk-neutral approach, in which we seek to minimize an exponential of the sum of costs. In particular, there has been work on risk-sensitive Markov Decision Processes (MDPs) { i.e., discrete-time problems with a nite or countable state space; this research is reviewed in 12]. We focus here on risk-sensitive control of partially observable MDPs, also known as Hidden Markov Models (HMMs). For such problems, an information state and dynamic programming equations are introduced in 1]. Structural results for the value function in the partially observed setting are provided in 7]. The average cost partially observed problem has been studied in 8]. As in 7], we study the structure of risk-sensitive controllers (policies), to attempt to understand the implications of the risk-sensitive cost on the behavior of the controller, and to compare the behavior of risk-sensitive and risk-neutral controllers. We will also study the behavior of the controller in the "large risk" (or "worst case") limit (see also 4]). In order to study these questions, we analyze a replacement problem involving the control of failure-prone units in production/manufacturing systems. This system is analyzed in 5] for the average cost risk-neutral criterion and in 7] for risk-sensitive nite horizon cost.